Search for: All records

World model based reinforcement learning (RL) has emerged as a promising approach for autonomous driving, which learns a latent dynamics model and uses it to train a planning policy. To speed up the learning process, the pretrain-finetune paradigm is often used, where online RL is initialized by a pretrained model and a policy learned offline. However, naively performing such initialization in RL may result in dramatic performance degradation during the online interactions in the new task. To tackle this challenge, we first analyze the performance degradation and identify two primary root causes therein: the mismatch of the planning policy and the mismatch of the dynamics model, due to distribution shift. We further analyze the effects of these factors on performance degradation during finetuning, and our findings reveal that the choice of finetuning strategies plays a pivotal role in mitigating these effects. We then introduce AdaWM, an Adaptive World Model based planning method, featuring two key steps: (a) mismatch identification, which quantifies the mismatches and informs the finetuning strategy, and (b) alignment-driven finetuning, which selectively updates either the policy or the model as needed using efficient low-rank updates. Extensive experiments on the challenging CARLA driving tasks demonstrate that AdaWM significantly improves the finetuning process, resulting in more robust and efficient . 

World model based reinforcement learning (RL) has emerged as a promising approach for autonomous driving, which learns a latent dynamics model and uses it to train a planning policy. To speed up the learning process, the pretrain-finetune paradigm is often used, where online RL is initialized by a pretrained model and a policy learned offline. However, naively performing such initialization in RL may result in dramatic performance degradation during the online interactions in the new task. To tackle this challenge, we first analyze the performance degradation and identify two primary root causes therein: the mismatch of the planning policy and the mismatch of the dynamics model, due to distribution shift. We further analyze the effects of these factors on performance degradation during finetuning, and our findings reveal that the choice of finetuning strategies plays a pivotal role in mitigating these effects. We then introduce AdaWM, an Adaptive World Model based planning method, featuring two key steps: (a) mismatch identification, which quantifies the mismatches and informs the finetuning strategy, and (b) alignment-driven finetuning, which selectively updates either the policy or the model as needed using efficient low-rank updates. Extensive experiments on the challenging CARLA driving tasks demonstrate that AdaWM significantly improves the finetuning process, resulting in more robust and efficient . 

Current biogeochemical models produce carbon–climate feedback projections with large uncertainties, often attributed to their structural differences when simulating soil organic carbon (SOC) dynamics worldwide. However, choices of model parameter values that quantify the strength and represent properties of different soil carbon cycle processes could also contribute to model simulation uncertainties. Here, we demonstrate the critical role of using common observational data in reducing model uncertainty in estimates of global SOC storage. Two structurally different models featuring distinctive carbon pools, decomposition kinetics, and carbon transfer pathways simulate opposite global SOC distributions with their customary parameter values yet converge to similar results after being informed by the same global SOC database using a data assimilation approach. The converged spatial SOC simulations result from similar simulations in key model components such as carbon transfer efficiency, baseline decomposition rate, and environmental effects on carbon fluxes by these two models after data assimilation. Moreover, data assimilation results suggest equally effective simulations of SOC using models following either first‐order or Michaelis–Menten kinetics at the global scale. Nevertheless, a wider range of data with high‐quality control and assurance are needed to further constrain SOC dynamics simulations and reduce unconstrained parameters. New sets of data, such as microbial genomics‐function relationships, may also suggest novel structures to account for in future model development. Overall, our results highlight the importance of observational data in informing model development and constraining model predictions. 

Key Points A new semi‐analytical spin‐up (SASU) framework combines the default accelerated spin‐up method and matrix analytical algorithm SASU accelerates CLIM5 spin‐up by tens of times, becoming the fastest method to our knowledge SASU is applicable to most biogeochemical models and enables computationally costly study, for example, sensitivity analysis 

Abstract Large across‐model spread in simulating land carbon (C) dynamics has been ubiquitously demonstrated in model intercomparison projects (MIPs), and became a major impediment in advancing climate change prediction. Thus, it is imperative to identify underlying sources of the spread. Here, we used a novel matrix approach to analytically pin down the sources of across‐model spread in transient peatland C dynamics in response to a factorial combination of two atmospheric CO 2 levels and five temperature levels. We developed a matrix‐based MIP by converting the C cycle module of eight land models (i.e., TEM, CENTURY4, DALEC2, TECO, FBDC, CASA, CLM4.5 and ORCHIDEE) into eight matrix models. While the model average of ecosystem C storage was comparable to the measurement, the simulation differed largely among models, mainly due to inter‐model difference in baseline C residence time. Models generally overestimated net ecosystem production (NEP), with a large spread that was mainly attributed to inter‐model difference in environmental scalar. Based on the sources of spreads identified, we sequentially standardized model parameters to shrink simulated ecosystem C storage and NEP to almost none. Models generally captured the observed negative response of NEP to warming, but differed largely in the magnitude of response, due to differences in baseline C residence time and temperature sensitivity of decomposition. While there was a lack of response of NEP to elevated CO 2 (eCO 2 ) concentrations in the measurements, simulated NEP responded positively to eCO 2 concentrations in most models, due to the positive responses of simulated net primary production. Our study used one case study in Minnesota peatland to demonstrate that the sources of across‐model spreads in simulating transient C dynamics can be precisely traced to model structures and parameters, regardless of their complexity, given the protocol that all the matrix models were driven by the same gross primary production and environmental variables. 

Abstract Soils store more carbon than other terrestrial ecosystems 1,2 . How soil organic carbon (SOC) forms and persists remains uncertain 1,3 , which makes it challenging to understand how it will respond to climatic change 3,4 . It has been suggested that soil microorganisms play an important role in SOC formation, preservation and loss 5–7 . Although microorganisms affect the accumulation and loss of soil organic matter through many pathways 4,6,8–11 , microbial carbon use efficiency (CUE) is an integrative metric that can capture the balance of these processes 12,13 . Although CUE has the potential to act as a predictor of variation in SOC storage, the role of CUE in SOC persistence remains unresolved 7,14,15 . Here we examine the relationship between CUE and the preservation of SOC, and interactions with climate, vegetation and edaphic properties, using a combination of global-scale datasets, a microbial-process explicit model, data assimilation, deep learning and meta-analysis. We find that CUE is at least four times as important as other evaluated factors, such as carbon input, decomposition or vertical transport, in determining SOC storage and its spatial variation across the globe. In addition, CUE shows a positive correlation with SOC content. Our findings point to microbial CUE as a major determinant of global SOC storage. Understanding the microbial processes underlying CUE and their environmental dependence may help the prediction of SOC feedback to a changing climate.